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Dynamic systems characterized by diversified evolutions are not only more flexible, but also more 
resilient to attacks, failures and changing conditions. This article addresses the quantification of the 
diversity of non-linear transient dynamics obtained in undirected and unweighted complex networks 
as a consequence of self-avoiding random walks. The diversity of walks starting at a specific node i 
is quantified in terms of a signature composed by the entropies of the node visit probabilities along 
each of the initial steps. Six theoretical models of complex networks are considered: Erdos-Renyi, 
Barabasi- Albert, Watts-Strogatz, a geographical model, as well as two recently introduced knitted 
networks formed by paths. The random walk diversity is explored at the level of network categories 
and of individual nodes. Because the diversity at successive steps of the walks tends to be correlated, 
principal component analysis is systematically applied in order to identify the more relevant linear 
combinations of the diversity entropies and to obtain optimal dimensionality reduction. Several 
interesting results are reported, including the facts that the transient diversity tends to increase 
with the average degree for all considered network models and that the Watts and Strogatz and 
geographical models tend to yield diversity entropies which increase more gradually with the number 
of steps, contrasting sharply with the steep increases verified for the other four considered models. 
The principal linear combination of the diversities identified by the principal component analysis 
method is shown to allow an interesting characterization of individual nodes as well as partitioning 
of networks into subgraphs of similar diversity. 

PACS numbers: 89.75.Fb, 02. 10. Ox, 89. 75. Da 



'There is a city where you arrive for the first time; and 
there is another city which you leave never to return. ' 
(Invisible Cities, I. Calvino) 

I. INTRODUCTION 

The diversity of dynamics plays a key role in most 
aspects of nature, which has ultimately resulted in a 
wealthy of species along evolution as well as a myriad of 
human cultural manifestations. Because of their capac- 
ity to represent discrete structures and scaffold dynamics, 
complex networks (e.g. [H, H, H, 0, Q ) have become the key 
paradigm in theoretical and applied studies in complex 
dynamic systems, finding applications in an impressive 
range of problems (e.g. [S!]). A great deal of the current 
attention in this area concentrates not only in charac- 
terizing the topological properties of networks (e.g. Q, 
but also in investigating how the latter constrains or even 
define dynamics unfolding in the networks (e.g. [1, 

With a tradition extending back over several decades, 
the study of the dynamics of random walks represents 
one of the main paradigms in statistical physics and dy- 
namical systems. Traditional random walks are usually 
performed by one or more agents choosing with uniform 
probability between the outgoing edges at each node. 
Therefore, random walks represent one of the least in- 
telligent ways to move in a network, involving no addi- 
tional criterion rather than uniform chance. Still, such a 
dynamics is directly related to the important linear dy- 
namics of diffusion (e.g. which plays an important 



role in a large number of natural dynamical processes 
(e.g. reaction-diffusion and Schrodinger equation). The 
dynamics of traditional, linear, random walks on com- 
plex networks has been investigated by several articles 
(e.g. B i, [M HH, [H El)- Several other types of ran- 
dom walks have also been considered in the literature 
(e.g. [3, [H, nil). For instance, the category of self- 
avoiding random walks represents a particularly inter- 
esting situation in which the moving agent is not allowed 
to return to nodes and/or edges. As such, self-avoiding 
walks arc can be directly associated to the paths existing 
in the networks. By vath, it is henceforth meant a se- 
quence of adjacent [2J| edges without repetition of node 
or edge. Paths are important because they provide the 
most effective way to connect the involved nodes (i.e. 
given M nodes, a path through them involves M — 1 
edges). In addition, unlike random walks, random self- 
avoiding walks — path-walks for short — are non-linear 
and necessarily finite in finite networks, because the mov- 
ing agent sooner or later has no way to proceed. The pos- 
sibility to use self-avoiding walks to sample networks has 
been investigated in [l^. Paths and self-avoiding walks 
have recently been explored as dual motifs of star connec- 
tivity building block of networks [3| and for char- 
acterization of networks (especially through the longest 
path) . The transient dynamics of self-avoiding walks 
in uniformly random, small world and scale free networks 
has been studied in [23, [2lj , with special attention placed 
on the average number of such walks. 

Random walks typically start from a node and pro- 
ceed j25| until some stopping condition is met (e.g. fixed 
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FIG. 1: Given a node i in a network (a), the entropies E{i, h) 
of the probabihties of visited nodes after the initial h steps 
(c) can be calculated by simulating several self-avoiding ran- 
dom walks starting from i. Therefore, a signature / (b)can 
assigned to each node i which expresses the diversity of paths 
obtained at each step h. 



number of steps or, in the case of self-avoiding walks, im- 
possibility to proceed further). In this work all walks 
are performed for a pre-specified number of 10 steps, 
as we are interested in the transient dynamics. Con- 
sider now that the starting node has been fixed and sev- 
eral self- avoiding random walks are performed from that 
node. One interesting question regards how such walks 
are composed and distributed. For instance, one may be 
interested in the length of these walks (e.g. [l^, [1q|). A 
question of special relevance which has received little at- 
tention from the literature regards the diversity of the 
obtained walks and path-walks. By diversity, it is meant 
how much the walks differ one another by incorporating 
distinct nodes and/or edges. In a previous approach to 
this problem, Herrero investigated the average number 
of self-avoiding walks defined in uniformly random and 
scale free networks. In the present article, the diversity of 
self-avoiding walks starting at a specific node i is quan- 
tified in terms of the entropies of the node probability 
visits after the first S steps along the walk after starting 
from each node i, giving rise to diversity entropy (see 
Figure [T]). Because of the non-linear nature of this type 
of walks and our interest in obtaining information about 
each individual walk in several types of structurally di- 
verse networks, the visit probabilities are estimated by 
performing several self-avoiding walks. 

Provided such a diversity of dynamics can be quanti- 
fied, ideally in terms of a single measurement, a series 
of interesting analyses can be performed at several levels 



(e.g. from individual node to network category levels). 
Indeed, the diversity of walks is immediately related to a 
large number of important theoretical and practical as- 
pects of complex networks structure and dynamics. To 
begin with, the cases in which the path-walks are found 
to be mostly similar (i.e. little diversity) imply that the 
agent had little choice during its motion, and therefore 
little path redundancy is present in the network, starting 
from that node. At the same time, such situations will 
also be characterized as being highly efficient as far as 
node coverage is concerned (relatively few edges are re- 
quired while visiting several nodes). Indeed, by recalling 
that all nodes in a path- walk must be distinct, a path- 
walk involving S +1 nodes will necessarily have S edges, 
which is the minimum number of connections required 
to connected those nodes. On the contrary, in case the 
path-walks are found to be strongly diverse, we can con- 
clude that the the progression of the agent is character- 
ized by great freedom of choice and variety of transient 
dynamics. Consequently, because the path- walks can not 
repeat nodes, we also have that the self-avoiding walks in 
this case will also involve several diverse nodes. There- 
fore, nodes with high diversity constitute natural can- 
didates as distributing sources (e.g. for information or 
mass) . It is also important to observe that the dynamics 
diversity of a node provides information which is comple- 
mentary to other measurements of network nodes. For 
instance, though diversity tends to be correlated with the 
node degree at the initial steps of the self- avoiding walks, 
such a correlation can be quickly lost as a consequence of 
the structural diversity at the progressive surroundings 
of the initial node. The walk diversity is also distinct 
from the betweeness centrality (e.g. [1, @) in the sense 
that chained nodes with high betweeness centrality will 
lead to low walk diversity. Therefore, diversity can be 
best thought as a novel measurement which can comple- 
ment previous approaches in the characterization of the 
structural properties of complex networks. 

Many are the interesting applications of such diversity 
studies to real-world problems. For instance, in case the 
random walks are used to model the acquisition of knowl- 
edge or cultural values by the agent (in this case each 
node represent a knowledge or cultural fact, e.g. pL6j), 
the diversity measurements can provide sound basis for 
discussing how diverse the development of agents start- 
ing from similar backgrounds but subsequently exposed 
to different information will be. Another particularly in- 
teresting application concerns the objective quantifica- 
tion of the diversity of life and species along phylogenet- 
ics, as well as geographical exploration. In addition to 
its potential for the objective characterization of dynam- 
ics performed in complex networks, the quantification of 
the diversity of random walks and path-walks can also 
provide valuable indications about the structure of the 
respective networks. For instance, in case all path-walks 
are identical, we have a chain of nodes extending from 
the starting node. Contrariwise, a high diversity implies 
the presence of redundancies in the network. 
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Because the diversity entropies at subsequent steps 
tend to be correlated, the statistical method known as 
principal component analysis (PCA) [l^ is systemati- 
cally applied in this work in order to decorrelate those 
measurements. The PCA method provides an optimal 
stochastic linear transformation in the sense of concen- 
trating the variation of the data along the first new ran- 
dom variables. In other words, the PCA transforms the 
original measurements into new features which are com- 
pletely uncorrelated one another. Because of the linear 
nature of PCA, the new obtained measurements corre- 
spond to linear combinations of the original features, 
weighted so as to optimize the concentration of variance 
along the first new variables. 

The manuscript starts by presenting the basic con- 
cepts, adopted network models, as well as the definition 
of the diversity entropy and some of its properties. The 
results are presented with respect to the analysis of whole 
network categories, individual networks, and individual 
nodes. 



II. BASIC CONCEPTS 



This section describes the basic concepts and methods 
used in this article, including network representation and 
characterization, the 6 adopted complex network models, 
the definition and estimation of the diversity entropy sig- 
nature, as well as the stochastic projections methods ap- 
plied in order to decorrelated the signatures and achieve 
dimensionality reduction. 



A. Complex Networks Representation and 
Characterization 

A unweighted and undirected complex network, 
formed by N nodes and E edges, can be fully represented 
in terms of its adjacency matrix K , which is symmetric 
and has dimension N x N. Each existing edge {i,j) im- 
plies K{i,j) = K{j,i) = 1, with K{i,j) = K{j,i) ^ 
indicating absence of that edge. Two edges are said to 
be adjacent whenever they share one of their extremities. 
A random walk corresponds to any sequence of adjacent 
edges (ii, 12); («2, ^a); ■ ■ • (ip-i, A walk which does not 
repeat any edge or node, henceforth called self-avoiding 
random walk, defines a path in the network. The length 
of a walk or path is equal to the number of its constituent 
edges. The shortest path between two nodes is defined 
as one of the paths between those nodes which has the 
smallest length. 

The immediate neighbors of a node i are those nodes 
which are connected to i through shortest paths of length 
1. The degree of a node is equal to the number of edges 
emanating from that node. The node degree averaged 
within a network is called its average degree. Extrem- 
ity nodes are henceforth understood as those with unit 



degree. As such, extremity nodes tend to determine the 
termination of many self-avoiding walks (no way back for 
the moving agent from that type of nodes). The cluster- 
ing coefficient of a node i is the ratio between the number 
of undirected edges between the immediate neighbors of 
i and the maximum possible number of undirected edges 
among those nodes. 



B. Complex Networks Models 

Six theoretical models of complex networks are consid- 
ered in the present work including four traditional mod- 
els — Erdos-Renyi (ER), Barabasi- Albert (BA), Watts- 
Strogatz (WS) and a geographical model (GG) — as well 
as two recently introduced knitted types of complex net- 
works — the path-transformed BA model (PA) and 
path-regular networks (PN). The ER, BA and WS net- 
works are grown in the traditional way (e.g. P, H, S 0, S] ) ■ 
The GG networks in this work are obtained by distribut- 
ing N nodes within a square with uniform probability 
and connecting all nodes which are closer than a mini- 
mal distance d. The PA and PN networks are obtained 
as explained in p^ : the PA networks {path-transformed 
BA networks) are obtained by star-path transforming all 
nodes in an original BA network and the PN {path regu- 
lar networks) model is easily obtained by defining paths 
involving all network nodes in random order and without 
repetition. 

All networks considered in this article have « 100 
and TO « 3 or m « 5 (to is the number of spokes in 
the added nodes in the BA model), with average degree 
(k) « 2m. The approximations are a consequence of the 
statistical variability of the models. For the same rea- 
son, the number of nodes N can vary slightly for the GG 
networks. Because the average degrees considered in this 
work are relatively large (well above the percolation crit- 
ical value for ER), most of the nodes in each network 
belong to the largest connected component, which has 
been considered for all the analyses reported in this arti- 
cle. The total of rewirings used in the WS case was equal 
to 0.1£:. 



C. Diversity Entropy and its Estimation 

The diversity entropy is the measurement used in this 
article in order to quantify the diversity of the self- 
avoiding random walks obtained for each node i at each 
step h. Let p{i,j, h) be the probability that a node j be 
visited after h time steps while moving from the starting 
node i. Once a self-avoiding walk is terminated (i.e. the 
moving agent can proceed no further), the moving agent 
is understood to remain at the final node and contribute 
to the probabilities and diversities for all remaining steps. 
The diversity entropy of node i can now be defined as: 



N 

E{i,h) = ~^p{i,j,h)log{p{i,j,h)) (1) 

Given the starting node i, E{i,h) can be understood 
as the diversity entropy signature for that node (see Fig- 
urefig:features). Each such signature can be transformed 
into a single value, e.g by taking the arithmetic or geo- 
metric average of its values considering all the steps h. 

Figure [5] illustrates several particularly relevant situa- 
tions regarding diversity entropy signatures. Because of 
the total absence of branches, the chain network in (a) 
yield a completely null diversity signature. This means 
total determinism in the sense that all self-avoiding ran- 
dom walks starting from i will be identical. The presence 
of a branch at step 3 in the structure in (b) implies the in- 
crease of the diversity entropy at this specific step. In the 
network in (c), the branch occurs at the first step, imply- 
ing diversity entropy E(i^ 1) = log{l/'i) « 1.1, which re- 
mains for the two following steps (i.e. h — 2 and 3). Ob- 
serve that the additional all-to-all connections between 
the nodes in the second and third steps have no effect 
in changing the respective diversity entropy, as they do 
not affect the respective probabilities p{i,j, 3). The situ- 
ation depicted in (d) involves self-avoiding random walks 
with different lengths, namely 1, 2 and 3. Because the 
moving agent is assumed to remain at its termination 
node, the diversity entropies do not change along the 3 
initial steps. Though this assumption implies eventual 
degeneracy such as obtaining the same diversity entropy 
signatures for the structures in (c) and (d), the distinc- 
tion between such cases can be easily accomplished by 
considering additional measurements such as the length 
of the walks. Finally, the situation shown in (e) involves 
converging connections at steps 1 and 2, which contribute 
to reducing the diversity of the random walks. Observe 
that the alternative assumption of removing the moving 
agent after it has reached a termination node would imply 
identical diversity entropy signatures for both structures 
in (d) and (e). 

Given a network with N nodes, the maximum diversity 
entropy obtained at any step is given when p{i,j,h) — 
l/N, implying 

N 

W ^ -l/N^log{llN)^log{N) (2) 

Figure [3] shows the maximum diversity entropies for 
several values of N . Therefore, as all networks considered 
in this article involves N 100, the diversity entropy is 
maximally bound toW = Zog(l/100) « 4.61. 

A particularly interesting situation occurs when each 
node at each level h leads exclusively to a constant num- 
ber (fc) of new nodes in the subsequent level h + 1 (see 
Figurell]). In this case, p{i,j,h) = l/{{k)^), so that 
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FIG. 2: Illustrations of diversity entropy signatures (the di- 
versity entropies are shown in bold): (a) as the paths from 
node i are all equal for this case, the entropies are null for all 
values of h; (b) the divergence of edges at /i = 3 implies the in- 
crease of the diversity entropy to l/Zo(;(3) « 1.1 at that level; 
(c) the presence of all-to-all connections between the nodes 
at steps 2 and 3 has no effect in increasing the entropies at 
level ft = 3; (d) because the moving agent remains at each 
terminal node after reaching it, the entropy does not change 
at the successive steps for this case; (e) converging edges (at 
the second and third steps in this particular example) can 
lead to decrease of the diversity entropy along h. 



E{i, h)^-J2 logm'')/ {kf = hlog{{k)) (3) 

Therefore, the diversity entropy will tend to increase 
(or remain null for (k) = 1) with h at constant rate 
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FIG. 3: The maximum diversity entropy which can be ob- 
tained for networks with A'^ nodes. 



log{{k)). This situation involves an infinite and com- 
pletely regular network (i.e. each node has the same de- 
gree {k As complex networks are often analyzed 
with respect to regular or nearly regular counterparts 
(e.g. ER model), it is useful to consider the above con- 
figuration as a reference. For instance, the situation in 
which the diversity entropy tends to increase almost lin- 
early with h along an interval can be understood as an 
indication that the network is mostly regular along that 
interval. However, it should be born in mind that linear 
increase of the diversity entropy can also be caused by 
other structural organizations in complex networks (i.e. 
constant increase of entropy does not necessarily implies 
network degree regularity, but the latter necessarily im- 
plies linear entropy increase). 

As the diversity entropy has been defined for each node 
i at each step /i, it provides an individual signature asso- 
ciated to each node (see Figure [1]), which can be valuable 
while investigating dynamics emanating from that node. 
However, it is often interesting to get an overall idea of 
the diversity entropy dynamics considering all the nodes 
in the networks. This can be immediately obtained in 
terms of the average and standard deviation of the diver- 
sity entropy at each step h, i.e.: 



1=1 

Var{E{h)} = ^ f;(i?(*, h) - {E{h))f (5) 
1=1 

aE(h) = +^Var{E{h)} (6) 

The algorithm adopted for picking a random path is 
simple and, given each starting node i, involves perform- 
ing AI self- avoiding random walks along the initial S 
steps (in this work, S — 10). Therefore, for each node 
i = 1, 2, . . . , iV, an accumulator array V of dimension 




FIG. 4; An example of a situation where the diversity entropy 
increases linearly with the steps h. 



S X N is kept in order to store the number of visits to 
each of the network nodes after starting from i. At every 
step h = 1, 2, . . . , along each of these M self-avoiding 
walks, the moving agent is found at a node j and the ele- 
ment V{h,j) of the accumulator vector is incremented by 
one. After completing the M self-avoiding walks start- 
ing from node i, the probability of visits to nodes can 
be estimated as p{i,j,h) — V{h,j)/M. Recall that once 
the moving agent reaches a termination node, it remains 
there for all remaining steps. For N = 100, the situation 
considered for all networks in this article, the probabili- 
ties have been experimentally found to have converged to 
less than 5% stability for M = 200, which is henceforth 
adopted. 



D. Optimal Dimensionality Reduction by 
Stochastic Transformations 



In several situations, especially for nearly uniform net- 
work (i.e. nodes having most nodes with similar proper- 
ties, such as degree), the diversity neighboring entropies 
along the steps of the signatures obtained for each node 
i will tend to be strongly correlated. Indeed, recall 
that each node i in the network will be mapped into 
a 10— dimensional feature vector (the diversity entropy 
signature), which is a relatively high dimensional space, 
impossible to be visualized. It is possible, and useful, to 
reduce the dimensionality of such measurement spaces by 
using optimal stochastic linear transformations such as 
principal component analysis and canonical projections 



6 



(e.g. I, Hi). 

The former of these approaches aUows the measure- 
ment space to be optimaUy projected into m < S* di- 
mensions, TO > 1, while maximizing the variation of the 
observations along the first transformed, new variables 
(e.g. [1, [121). The transformed variables, which arc lin- 
ear combinations of the original measurements, are guar- 
anteed to be completely uncorrelated. The latter trans- 
formation (i.e. canonical projections) allow the measure- 
ment space to be optimally projected into a smaller di- 
mensional space while maximizing the separation of the 
categories of observations, in the sense of maximizing the 
interclass variation and minimizing the intraclass disper- 
sion (e.g. 0, HI). In both cases, the resulting trans- 
formed variables are linear combinations of the original 
measurements. 

Though still rarely applied in complex network re- 
search, such optimal stochastic transformations can be a 
real help in organizing and simplifying the analysis and 
classification of complex networks (e.g. 0]). In this work, 
the principal component approach is used in order to 
decorrelate the diversity entropy signatures obtained for 
each of the nodes in a given complex network, while the 
canonical projections method is applied in order to obtain 
visualizations of the distribution of several realizations 
of 6 different types of complex network models. Addi- 
tional information about the canonical projections analy- 
sis, which is mathematically more sophisticated than the 
principal component approach, can be found in |23 | . 
The principal component analysis, used to decorrelated 
the diversity entropies in this work, is described as fol- 
lows. 

Let / be the feature vector containing the S measure- 
ments obtained for each observation i = 1, 2, . . . , A^. In 
the present work, each node i (an observation) is mapped 
into a diversity entropy signature (the feature vector) of 
dimension Sxl. The elements C{i, j), i, j = 1, 2, . . . , S" of 
the covariance matrix of such a dataset can be estimated 
as 



where to < 5, to > 1, / and g have respective dimen- 
sions 5' X 1 and m x 1. Therefore, the new measurements 
(transformed variables) belong to a space of reduced di- 
mensionality m < S. The new measurements associated 
to the largest eigenvalues are called the main variables 
or components. Observe that each of the new measure- 
ments is a linear combination of the original measure- 
ments, while the eigenvalues 7^, i = 1, . . . , to, correspond 
to the variances of the new measurements in g. So, it is 
reasonable to include in the transformation matrix only 
the eigenvectors associated to eigenvalues which are par- 
ticularly large, in order to encompass the greatest part 
of the original variation of the observations. Also, ob- 
serve that the relative weight of each of the original mea- 
surements used in the linear combinations defining the 
new variables can provide an indication about the impor- 
tance of the respective original measurements. Because 
the feature vectors considered in this work have all the 
same nature and potential dynamic range (recall that all 
the elements of the diversity entropy signature are en- 
tropies vary between and log(N)), there is no need for 
preliminary standardization of the original measurements 
(e.g. i). 

III. RESULTS AND DISCUSSION 

The diversity entropy methodology has been applied 
at the level of network categories and individual nodes. 
In the former case, the overall diversity was character- 
ized with respect to the average and standard deviation 
of the diversity entropies obtained for each realization of 
the networks. The latter investigation targets the estima- 
tion of the diversity entropy signature at the individual 
node level, which allows the partitioning of each network 
into subgraphs of similar diversity. These two types of 
investigations, at the network and individual node levels, 
are described in the respective following sections. 



N 
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-Y.^v{i)~tiMj)~H) (7) 



where is the average of z;(a), a = 1, 2, . . . , S'. Ob- 
serve that C{i,i) = Var{i) whenever i = j. Also, we 
have that the covariance matrix C is necessarily sym- 
metric. 

Let 7i, i = 1, 2, . . . , 5 be the eigenvalues of the covari- 
ance matrix, ordered so that 71 > 72 > . . . > 7s , and let 
qi be the respectively associated eigenvectors. The prin- 
cipal component analysis can be obtained by performing 
the following stochastic linear transformation 



9 = 



92 



(8) 



A. Network Level 

We start our diversity investigation by looking at the 
averages and standard deviations of the diversity entropy 
signatures obtained for the realizations of each of the 6 
considered network models. More specifically, a total of 
50 realizations were performed for each of the six complex 
network models considering N — 100 and two average 
degrees: (a) (fc) = 6 (i.e. to = 3) and (b) (fc) = 10 (i.e. 
TO = 5). For each of such realizations, 200 random path- 
walks were performed starting from each of the nodes, 
and the respective entropies E(i^ h) were estimated for 
h = 1, 2, . . . , 10. The average {E{h)) and standard de- 
viations <JE{h) of these diversity entropy values were ob- 
tained for each of the 50 network realizations for each of 
the 6 considered models and are shown in Figures [5] and [5] 
respectively to to = 3 and m — 5. 

A series of interesting results can be inferred from the 
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curves in those figures. First, observe that markedly dis- 
tinctive curves and standard deviations were obtained for 
the networks belonging to each of the considered models. 
Two general behaviors can be distinguished in both fig- 
ures: the steeper increase of the diversity entropy with h 
obtained for the ER, BA, PN and PA modes as opposed 
to the more gradual increase verified for the WS and GG 
models. These two types of transient dynamics can be 
observed for both to = 3 and to = 5. In the case of the 
transient evolutions observed for the ER, BA, PN and 
PA models, the diversity entropy tended to reach sta- 
bilization near a higher plateau (with diversity entropy 
approximately equal to 4.2) after the three or four ini- 
tial steps. This suggests that the self-avoiding paths in 
these networks tend to reach most nodes after just a few 
steps. The more gradual increase of entropy observed for 
the WS and GG models indicates that the moving agent 
takes substantially more time to cover a smaller portion 
of the nodes during the transient dynamics. This is a 
consequence of the fact that, though nearly regular (i.e. 
similar degrees for all nodes), these two types of net- 
works are characterized by having pairs of nodes which 
are either connected through many short paths (adjacent 
nodes) or virtually unconnected. More informally, given 
two nodes i and j of a network, the adjacency between 
them can be quantified in terms of the number of short 
(i.e. up to a maximum length) paths interconnecting 
those nodes; the higher this number, the more adjacent 
the pair of nodes is. 



Another interesting result regards the maximum en- 
tropy values, reached for large values of h. Except for the 
GG model, all other types of networks tended to entropies 
around 4.2 for both m — i and m ~ 5. Interestingly, 
quite similar limiting entropy values have been obtained 
for BA and PN networks irrespectively of the average de- 
gree. By comparing the respective curves in the two fig- 
ures, it becomes clear that the higher average degree (i.e. 
TO = 5, implying (fc) = 10) tended to reduce the standard 
deviations along the plateaux of the BA, WS, PN and 
PA cases. Similar standard deviations can be observed 
for the GG case, while higher deviations now characterize 
the plateaux for the ER models. Though at first surpris- 
ing, the decrease of the standard deviations observed for 
most models is ultimately a consequence of the fact that 
increasing the average degree of finite networks tends to 
make them more regular, implying in more self-avoiding 
paths covering the same set of nodes. Observe that at 
the extreme situation in which the network is fully con- 
nected, all path- walks will involve all nodes, implying 
null variance of the diversity entropy. The higher aver- 
age degree also tended to change the shapes of the curves 
by implying a steeper increase (especially for the second 
step) along the initial step, which is also a consequence 
of the above observed regularizing effect. 

Another interesting result which is evident from Fig- 
ures [S] and [H] are the markedly distinct standard devia- 
tions obtained for each model. Confirming previous in- 
vestigations fl§L [l9| , the PN model presented the more 
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FIG. 6: The average ± standard deviations of the diversity entropies obtained for each of the networks considered for each of 
the six complex networks models assuming A'^ = 100 and m = 5 (i.e. (k) = 10). 



regular features, with almost null standard deviations of 
the diversity entropies for either m = 3 or m = 5. Al- 
lied with the fast increase of diversity entropy exhibited 
by this model, the extremely low variance of diversity 
entropies makes of the PN a choice model for achieving 
high and uniform diversity signatures. To a great extent, 
such properties favoring diversity are a consequence of 
the fact that, unlike the WS and GG models, any pair 
of nodes in the PN structures tend not to be adjacent in 
the sense of being interconnected by many short paths. 
Recall that the ER, WS, GG and PN are all network 
models characterized by high degree regularity, so that 
what makes them so different regarding diversity is ul- 
timately the adjacency between pairs of nodes, which is 
optimally broken in the PN model. 

In order to complete our analysis of the diversity en- 
tropy signatures for networks belonging to the 6 distinct 
considered models, we now apply the canonical projec- 
tion method (see Section IIIPp . We use this method to 
project the original 10— dimensional entropies space into 
a 2— dimensional space so as to maximize the separation 
between the clusters of networks belonging to each cat- 
egory. Figure [7] shows the cluster distributions obtained 
for m = 3 (a) and m — b. 

It is clear from Figure [71 where vl and v2 correspond 
to the two principal canonical variables, that the 6 cate- 
gories of networks yielded two supergroups: one formed 
by {GG, WS} and the other by {ER, BA, PN, PA} (ob- 
serve the different ranges of values for the two axes). 



This is in complete agreement with the two main types 
of diversity dynamics identified for those networks (i.e. 
steeper and more gradual increase of the entropies). In 
addition to confirming that previous result, the canoni- 
cal projections showed that the ER and PN have marked 
similarity between their diversities (i.e. the clusters for 
these two models were mapped nearby in the projected 
space). The smallest dispersion of the diversities ob- 
tained for the PN model are clearly reflect in the dense 
cluster obtained for that category of networks. Interest- 
ingly, the ER and PA clusters tended to change positions 
considerably for m = 3 and m = 5. 

Additional results can also be obtained by consider- 
ing the weights of the original measurements in the lin- 
ear combinations defining the two canonical variables vl 
and v2, shown in Tabic HI We concentrate attention 
on the absolute values of the weights in Table |T1 In 
the case m = 3, we have that the two first canonical 
variables vl and v2 are mostly affected by the diver- 
sity entropies for h = 1,2,4,7,8,9 and 10, which are 
the main measurements responsible for the optimal sep- 
aration between the 6 models in the case m = 3. The 
most important measurements in the composition of the 
two canonical variables for m — 5 are the diversities ob- 
tained for h — 1,2, 3, 4, all of which with weights larger 
than 0.5. These measurements, which correspond to the 
initial steps of the walks, greatly contributed to the sepa- 
ration of the 6 network categories. The dominant contri- 
bution of the initial entropies is reasonable because most 
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FIG. 7: The clusters of networks, for m = 3 (a) and m — 5 (b), after being canonically projected from 10 to 2 dimensions so 
as to maximize the separation between the six categories of networks. 



the diversity signatures tend to become stable and sim- 
ilar after 3 steps in the case of to = 5 (recall that the 
diversity entropies rise more abruptly for to = 5 than for 
TO = 3). Such results obtained for m = 5 indicate that, if 
the main purpose of the analysis is to separate the 6 net- 
work types, it is mostly enough to consider the 4 initial 
entropies in each signature. 



m — 3 


m — 5 


vl 


v2 


vl 


v2 


0.45 


0.37 


0.51 


0.12 


-0.03 


-0.36 


-0.29 


-0.58 


-0.21 


-0.01 


-0.63 


-0.14 


-0.52 


0.18 


-0.16 


0.57 


-0.08 


0.19 


0.07 


0.23 


-0.09 


-0.36 


0.31 


0.37 


0.40 


-0.05 


-0.05 


0.02 


0.32 


0.39 


0.22 


-0.16 


0.18 


0.42 


0.19 


-0.30 


-0.42 


-0.45 


-0.19 


-0.12 



TABLE I: The weights of the original measurements assigned 
by the canonical projections method in order to best separate 
the 6 categories of complex networks in the 2— dimensional 
projections for m = 3 and m — 5. 



B. Individual Node Level 

Having investigated how the diversity entropies behave 
in each of the 6 considered network categories, we now 
turn our attention to the diversity entropy signatures ob- 



tained at the level of individual nodes. In order to do so, 
we selected a network of each type and obtained the re- 
spective signatures shown in Figure [5] and [51 with respect 
to TO = 3 and m — 5. Similar signtures were obtained 
for other realizations of each of the 6 types of networks. 
The geographical network considered in this analysis is 
shown in Figure [T^ 

Most of the results and explanations presented in the 
previous analyses at the network level can be immedi- 
ately extended to the signatures in these curves. First, 
the dispersion of the signatures tend to be smaller for 
TO = 5 than for m = 3. Very similar signatures were ob- 
tained for all nodes in the PN networks, which confirms 
the distinctive regularity of this model. The two types of 
transient dynamics, namely steeper for the ER, BA, PN 
and PA networks and more gradual for the WS and GG 
structures, were again observed. The most interesting ad- 
ditional information provided by the presentation of the 
individual node signatures regards the relative dispersion 
obtained for each case. Observe that particularly distinct 
diversity signatures were obtained for the GG structure. 
This is mainly a consequence of the higher structural 
modularity and overall adjacency found in this type of 
network (see Figure fT2)) . 

Additional insights about the measurement structure 
in each of the networks can be obtained by applying prin- 
cipal component analysis to each of the datasets in Fig- 
ures [8] and [8] in order to obtain 2— dimensional visualiza- 
tions of the distribution of the respective diversity sig- 
natures. The projections obtained for to = 3 is shown 
are Figure [TU] (the projections for to = 5 are very simi- 
lar and are not shown in this article). Recall that each 
point in these plots corresponds to each of the nodes in 
the respective sample network. Distinct clusters, all of 
which completely uncorrelated, were obtained for each 
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of the networks. Note the presence of outhers for the 
cases ER, BA and GG. The distribution obtained for the 
GG structure presented the largest dispersion of points. 
Contrariwise, the projection of the diversity entropy sig- 
natures for the PN network yielded the most compact 
cluster, confirming once again the enhanced regularity of 
this type of network. The dispersions of the ER, BA, WS 
and PA networks are similar. 

Additional insights about the influence of the measure- 
ments (i.e. the diversity entropies) in the definition of 
the clusters in Figure [TO] can be obtained by considering 
the respective weights of the original measurements in 
the linear combinations defining the two main principal 
variables peal and pca2, which necessarily resulted com- 
pletely uncorrelated. Observe that the variance of peal 
is much wider than that ofr v2 in all cases. Such weights 
are given in Table [Til It is clear from the values in this ta- 
ble that the initial 3 or 4 diversity entropies are the most 
relevant measurements in all situations, except for the 
WS and GG cases. The special importance of the first 
entropies is a consequence of the fact that the signatures 
tend to be more different at such initial steps, getting 
nearly constant once they reach the plateaux. The dis- 
tinct composition exhibited by the first principal variable 
in the case of the WS and GG networks reflects that the 
individual signatures remain distinct even after 3 or 4 
steps. Because of its largest variance, the first principal 
variables ire particularly relevant as a single quantifica- 
tion of the individual node diversities. In the case of 
the WS and GG networks, this variable corresponds very 



closely to the arithmetic mean (to a multiplicative fac- 
tor). This summarizing measurement is henceforth called 
the overall diversity of each node. 

In order to conclude our investigation of the diversity 
entropy signatures at the individual node level, we con- 
sider the GG network chosen for the above examples (see 
Figure [T2)) for a more systematic investigation of the di- 
versities. The choice of this type of network is justified 
because it is the only case among the considered cate- 
gories which incorporates the spatial positions of each 
node and because this type of network tends to exhibit 
structured modularity (i.e. spatial and topological com- 
munities). 

Because the first principal variable has been verified to 
correspond very closely to the arithmetic average of the 
diversity entropies for all network types, we adopt this 
value in order to summarize the diversity of each node 
in the chosen network. Figure [TIT a) shows an enlarged 
version of the PGA projection of the diversity entropies 
obtained for this geographical network. Because of the 
right-skewed distribution of the density of the points in 
this projection, we consider a new projection obtained by 
taken the exponential of the first PGA variable, hence- 
forth represented as exp{pcal). This new projected dis- 
tribution is shown in Figure [TlT b) . A more uniform dis- 
tribution of points is now obtained. We now subsume 
the new variable exp{pcal) into 9 intervals identified by 
the colors in Figure [TTT bV Observe that, because peal 
is very close to the arithmetic average of the diversity 
entropies for the various values of /i, the diversity of the 
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0.01 
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0.01 
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0.01 
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0.26 


0.26 


0.33 


0.18 
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0.12 


0.01 


0.00 


0.02 


-0.18 


-0.01 


-0.01 


0.22 


0.20 


0.33 


0.21 


-0.02 


0.07 


0.00 


-0.02 


0.02 


-0.21 


0.03 


-0.02 


0.17 


0.13 
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0.06 


0.00 


0.01 


0.01 
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0.01 


0.02 
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TABLE II: The weights of the original measurements assigned by the principal component analysis method in order to completely 
decorrelate the diversity entropy signatures for each of the networks representing each of the 6 categories of networks. 



nodes increase from left to right in both Figures [TlT a) 
and (b). Figure fT2l shows the original GG structure with 
its nodes colored according to the overall diversity inter- 
vals in Figure fTlTb) . 

A series of interesting results can be identified. First, 
observe that the nodes belonging to more external struc- 
tures tend to present the smallest overall diversities (in 
black). All extremity nodes (i.e. nodes with degree 1) 
are characterized by the smallest diversity. At the same 
time, the more densely connected groups of nodes tended 
to exhibit higher diversity, with node 29 presenting the 
largest diversity in this network. Indeed, the dense con- 



nectivity of the communities of nodes (to the left and 
right of node 29) allow several self-avoiding random walks 
to evolve from that node. Nodes 28, 34 and 43 exhibit 
the second highest diversity. Though such facts seem to 
suggest a strong correlation between diversity and node 
degree (see Q for an investigation about the correlation 
between the frequency of visits to nodes and their de- 
gree), this is not the case. As is clear from Figure [T3l 
which presents the scatterplot obtained by considering 
the node degree and overall diversity for the network in 
Figure fT^ these two measurements only a relatively weak 
positive correlation can be observed between these two 



12 




FIG. 10: The two-dimensional projections of the diversity entropy signatures obtained for each of the nodes in a sample of each 
of the six complex networks models assuming A'' = 100 and m = 3 (i.e. {k} = 6). 



measurements. Indeed, several nodes with high degree 
(i.e. hubs) in the GG structure — including nodes 11, 
61, 62, 70, 73, 75, 78 — do not have high diversity. On 
the other hand, node 45, with a high diversity, has a very 
low degree (2). Such as nodes 34 and 43, this node im- 
plements a bridge between the two main communities in 
this GG structure. At the same time, nodes 28, 34 and 
43 are hubs characterized by high diversity. Such a weak 
correlation between node degree, clear from the scatter- 
plot in Figure Q , and diversity can be accounted by the 
fact that the degree is an exclusively local property of 
a node, while the diversity is affected by the topological 
properties of many other surrounding nodes. As a further 
illustration of the more intricate nature of diversity, con- 
sider the two following very simple networks: (i) a hub 
whose adjacent nodes are weakly interconnected; and (ii) 
a hub with strongly interconnected adjacent nodes. In 
the former situation, after the moving agent leaves the 
hub to an adjacent node j, it can proceed only to the few 
nodes connected to j, implying small diversity at h — 2 
and subsequently. Contrariwise, because of the many in- 
terconnections between the nodes adjacent to the hub 
in situation (ii), many more self-avoiding walks will be 
possible, increasing the diversity at /i = 2 and beyond. 
Therefore, the diversity is also affected by the cluster- 
ing coefRcient around each of the nodes along the self- 
avoiding walks. Observe also that though most pair of 
adjacent nodes tend to present similar diversities, this is 
not necessarily guaranteed (see, for instance, nodes 29 



and 16). 

Another interesting aspect regards the possible rela- 
tionship between diversity and community structure. As 
suggested by the situation of node 45, it could be conjec- 
tured that nodes placed between two communities would 
tend to present higher diversity, as the self-avoiding walks 
emanating from such nodes could proceed freely inside 
both communities. At the same time, small communities 
such as that in the left-hand side of the graph in Fig- 
ure [T^] would tend to have nodes exhibiting low diversity. 
Because diversity is not directly related to betweeness 
centrality, additional studies are necessary in order to in- 
vestigate more precisely how diversity and communities 
are related. 

The quantification of the overall diversity of each node 
allows many interesting practical interpretations and ap- 
plications. For instance, in case the moving agent is at- 
tacking the network, the greatest disruption will be ob- 
tained in the cases in which it starts from high diversity 
nodes, such as 29 and 43. This would be true even if in 
cases the moving agent destroys the nodes after visiting 
them. At the same time, distribution of mass of infor- 
mation would be most effectively performed by allocating 
the sources to nodes with high diversity. Another inter- 
esting applications would be to consider the spatial explo- 
ration of the network while starting from different nodes. 
Exploring agents starting from low-diversity nodes will 
have to invest much more efforts (i.e. steps) in order to 
explore the network nodes than those starting at higher 
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diversity nodes. Immediate analogies can be drawn with 
WWW exploration and knowledge acquisition [l^ , where 
each node represents a piece of knowledge which are ac- 
quired by the moving agent as it moves through the net- 
work. In addition, edge or node failures or attacks tak- 
ing place at low-diversity regions of the network will have 
higher changes of causing major disruptions to the con- 
nectivity. 

It is important to observe that experiments performed 
considering diversity entropy signatures defined by tra- 
ditional random walks (i.e. with possibility of repeating 
edges and nodes) have led to substantially less intuitive 
and informative results regarding the structure of the an- 
alyzed networks. 



IV. CONCLUDING REMARKS 

Several systems in the real-world involves non-linear 
transient evolution after starting from well-defined, spe- 
cific states. Examples of such dynamics are numerous 
and include WWW navigation, Internet routing, evolu- 
tion of knowledge and cultural acquisition (e.g. dis- 
ease spreading, as well as evolutionary processes leading 
to new species (phylogenetics), to name but a few. Once 
the underlying system has been properly represented as 
a complex network, transient dynamics can be inves- 
tigated by performing diverse types of random walks. 
Though several works have been reported involving the 
traditional random walk, where nodes and edges can be 
visited more than once, relatively few approaches have 
concentrated attention in non-linear transient dynam- 
ics obtained by performing self-avoiding random walks 
(e.g. [ll,l23j[lH)- One particularly interesting aspect of 
such a kind of transient dynamics concerns its necessar- 
ily finite nature (as opposite to traditional random walks) 
and more purposeful exploration and coverage of the net- 
works, in the sense of connecting nodes with the smallest 
number of edges. Therefore, the characterization of such 
a type of dynamics can provide valuable resources not 
only for understanding the structure of networks, but 
also for quantifying important dynamics properties such 
as node coverage and redundancy/resilience. 

The current work focused attention on the issue of how 
diverse are the self-avoiding random walks performed in 
different types of networks after starting from different 
nodes. Because we wanted to obtain information about 
the diversity along the several steps while moving from a 
specific starting node, a simple sampling algorithm was 
used in order to estimate the probabilities of visits to 
nodes required for the diversity entropy calculation. In 
such a way, diversity entropy signatures can be estimated 
for every node in different types of networks. The main 
contributions of the current article are listed and briefly 
discussed in the following: 

Definition of the diversity entropy signature: 
An objective way to quantify the diversity of self-avoiding 
random walks has been described which takes into ac- 



count the diversity along several subsequent steps along 
the walk, after starting from an individual node i. Such 
an approach allows the discrimination, for instance, be- 
tween walks which are diverse only at the initial steps 
and walks which are diverse only at the later steps. The 
estimation of the diversity in terms of the entropy of the 
probabilities of visits to nodes provides a natural and in- 
tuitive choice for measuring the diversity. A simple but 
effective sampling algorithm has been suggested which 
can estimate the probability of visits to nodes at each 
step along the self-avoiding walks. The diversity entropy 
signature can be used for several purposes, including the 
study of network categories and the properties of nodes 
in specific networks. Both these possibilities have been 
explored in this article. Though potentially affected by 
several other topological measurements (e.g. node de- 
gree, clustering coefficient and betweeness centrality), 
the diversity is not necessarily correlated to any of these 
features, therefore providing complementary information 
about the topology and dynamics of complex networks. 

Characterization of the diversity of several 
types of networks: The diversity entropy signature has 
been used in order to investigate the general diversity of 
categories of networks. In this work, we considered 6 di- 
verse and representative theoretical models: Erdos-Renyi 
(ER), Barabasi- Albert (BA), Watts- Strogatz (WS) and 
a geographical model (GG) — as well as two recently in- 
troduced knitted types of complex networks [l^ — the 
path-transformed BA model (PA) and path-regular net- 
works (PN). The diversity entropy signatures were esti- 
mated for all nodes in each realization of each of these 
models, and summarized in terms of their average and 
standard deviation for each network. A series of inter- 
esting results were obtained, including the verification of 
the increase of diversity with the average node degree, 
the existence of two types of transient dynamics (steep 
and gradual increase along the steps), the high disper- 
sion of diversity observed for the WS and GG models, as 
well as the surprising uniformity of the diversity signa- 
tures for the PN networks. The latter model provides one 
of the most effective structure ensuring steep increase of 
diversity for all nodes and can be used in the design of 
several practical systems such as information dissemina- 
tion/exploration and transportation systems. 

Definition of Adjacency as a principal cause of 
diversity: The two types of signature regimes identified 
among the 6 considered theoretical models are at least 
partially accounted by the degree of adjacency between 
any pairs of nodes in each network. While the standard 
adjacency between two nodes simply states that there is 
at least one edge connecting them, the concept of degree 
of adjacency has been considered in this work in order 
to take into account the adjacency implemented through 
longer connections. More specifically, the degree of ad- 
jacency between any two nodes in a network at a given 
path-length is understood as the number of paths of that 
length connecting the two nodes. By using such a con- 
cept, it becomes possible to discriminate between oth- 
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erwise degree regular models such as ER, WS, GG, PN 
and PA. More specifically, because of spatial constraints, 
both the WS and GG structures have most pairs of nodes 
exhibiting higher degree of adjacency for several path- 
lengths. Such a local connectivity enhances the chances 
of termination of the self-avoiding walks, therefore reduc- 
ing considerably the diversity entropies. Such an effect 
has been clearly identified in the obtained results, leading 
to the identification of two regimes of transient evolution 
of the diversity. 

Use of sound multivariate statistics to decorre- 
lated the entropies: Because the diversity entropies 
at subsequent steps tend to be highly correlated, espe- 
cially for more regular networks, it becomes essential to 
extract the most representative information from the sig- 
natures by decorrelating the entropies at each step. In 
this work we have applied two sound and established op- 
timal methods for dimensionality reduction for obtain- 
ing more representative projections of the diversity sig- 
natures. The canonical projection methodology, which 
performs dimensionality reduction in order to optimally 
maximize the separation between the clusters produced 
by each category of networks, was applied in this work in 
order to emphasize the relationship between the diversity 
structure of the 6 considered models. The weights as- 
signed to each original measurement in the case of m = 5 
by this transformation confirmed the special information 
of the initial 3 or 4 diversity entropies for the discrimina- 
tion between the considered models. The principal com- 
ponent analysis approach was applied in order to decorre- 
late the entropy signatures obtained for individual nodes 
in specific networks. In most cases, the first principal 
variable was found to be mainly defined by the initial en- 
tropies. In the case of the WS and GG models, the first 
principal variable corresponded very closely to the arith- 
metic average of all entropies. This variable has been 
used to summarize the diversity of individual nodes of 
specific networks into a single measurement, called over- 
all diversity. Observe that the use of the arithmetic av- 
erage as a single descriptor in the case of the WS and 
GG models has not been imposed a priori, but estab- 
lished as a consequence of optimal decorrelation between 
the several diversity entropies. The second principal vari- 
able was found to be more strongly affected by the first 
diversity entropies, confirming the importance of these 
features. 

Characterization of the diversity of several 
types of networks at the individual node level: 

The diversity entropy signature was also investigated at 
the level of individual node for an example network from 
each of the 6 considered types of structures. Such an 



analysis led to results similar to those obtained for the 
global analysis. By using the principal variable as a sin- 
gle quantification of the diversity of the individual nodes, 
we were able to study in more detail a sample of GG net- 
work, which provides the spatial position of the nodes 
and exhibit community structure. The overall diversity 
was estimated for each node and 9 intervals of diversity 
were defined by binning the exponential values of the 
overall diversity, which allowed a more uniform distribu- 
tion of the measurements. The nodes of the network were 
then identified (colored) according to such intervals, lead- 
ing to an interesting partitioning of the network in terms 
of the respective diversities. A series of interesting results 
were obtained. First, the low-diversity nodes tended to 
appear at the borders of the network, often involving an 
extremity node. Interestingly, the own concept of diver- 
sity can be used to define; the network border nodes as 
corresponding to those which have little access to the 
remaining network. Second, the more densely intercon- 
nected nodes tended to present high diversity. However, 
no strong correlation has been identified, at least for the 
considered GG network, between node degree and diver- 
sity. Possible relationships between diversity and com- 
munity structure have also been identified. 

Such diverse results are not interesting by themselves 
regarding several aspects of complex network research, 
but also open several possibilities for future exploration, 
including but not being limited to: (i) study of other 
types of walks, such as preferential; (ii) applications to 
the characterization of real-world networks; (iii) study of 
scaling effects, especially with A''; (iv) consider the length 
of the self-avoiding walks as complementary information 
about the structure of the networks; (v) investigate more 
systematically the relationship between the degree of ad- 
jacency and the diversity, especially the possibility if the 
latter can be predicted by the former; (vi) devise net- 
work growth algorithms based on diversity constraints; 
(vii) use of the diversity to identify particularly weak 
and stronger points in networks (e.g. with respect to 
resilience or distribution) and try to enhance such sit- 
uations; and (viii) investigate further the possible rela- 
tionship between diversity and communities, especially 
regarding possible influences between diversity and be- 
tweeness centrality. 
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FIG. 11: The PCA projection obtained for the chosen GG network (a), and its transformation (b) obtained by taking the 
exponential of the first PCA variable, i.e. expijpcal). The colors identify each of the intervals of the exp(pcal) range. 
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FIG. 13: The scattcrplot obtained by considering the ovcrah diversities of the nodes in the GG structure and ther respective 
degrees. Only a weak correlation can be identified between these two measurements. 



